Search engines utilize information related to (or associated with) Websites to generate a list of search results by matching or otherwise comparing that information to search terms or keywords provided by a user. The information about or associated with a Website is derived from one or more of the webpage contents, HTML tags, or other data associated with a Website or webpage, and is typically referred to as metadata. Because metadata related to a Website or webpage is communicated to and used by search engines, it has been found that the metadata may be used to improve the accuracy of the search results produced by the search engines, and such techniques are often referred to as search engine optimization (SEO).
To obtain the “best” search results (i.e., results having the greatest likelihood of being of value to a user in terms of identifying a Website or webpage of potential interest), it has been found to be desirable to construct a “catalog” of the metadata manually, thereby reflecting attributes of the contents of the Website or webpage; for example, for an eCommerce platform Website or webpage, these attributes may include brands, colors, styles, and apparel sizes. The metadata may then be subjected to further evaluation or processing, such as by a curation process or operation. However, changes to the Website contents (as may arise due to sales, promotions, bundling of items, changes in availability, etc.) may not be reflected in the metadata unless the metadata catalog construction and curation are performed manually and at regular enough intervals (i.e., regularly enough to reflect the majority of the changes or those expected to be most significant). Unfortunately, in conventional systems and applications, the metadata regarding a Website tends to not accurately reflect the latest (most current) information regarding the Website and its contents (or at least the information that may be used to characterize a Website or webpage for purposes of responding to a search query) without the use of regular review and manual input. This means that the search results returned in response to a query or search may not be as complete or as accurate in the absence of manual curation or another form of regular updating. Further, unless the curation or updating is performed regularly and relatively frequently, some users may still receive stale search results, or at least sub-optimal results. This can be a serious problem; in the case of an eCommerce website, such a result may lead to missed sale opportunities and/or dissatisfied customers.
Embodiments of the inventive system and methods are directed to overcoming the limitations associated with conventional approaches to updating and maintaining Website or webpage metadata, both individually and collectively.